3D Graphics rendering systems, such as gaming PCs and gaming devices follow a standard architecture that typically includes:                1. CPU subsystem—it includes the main processor, memory and storage        2. Graphics subsystem—it includes the graphics processor (GPU) and associated memory        3. A Display subsystem that is connected to the GPU        
The CPU subsystem and the GPU subsystem are connected through a high speed bus, such as PCI, AGP or PCI-Express. The GPU subsystem is connected to the Display through another high speed interface such as HDMI, DVI, or Display Port. The role of these components can be thought of as the CPU being responsible for describing the content at an abstract level and the GPU is responsible for rendering the content in a pixel form. The Display is responsible for visually showing the pixels to the user.
Typically, the main program generating the graphics, such as a game program, is run on the CPU where the game program listens to user input from keyboard or game pad. The game program executes the game logic and then sends commands to the GPU telling the GPU how to create a picture (also called as frame) that will be shown on the Display. This process is repeated several times every second to create an appearance of smooth motion on the Display. Typically it is repeated 30 times a second. This figure is also knows as refresh rate.
It is GPU's job to execute the commands sent by the CPU. Commands can be roughly categorized as “simple commands” that GPU can execute by itself, “indirect commands” that refer to data residing in the CPU's memory (known as System Memory), or commands that read the data generated by the GPU.
Typically the volume of data going from the CPU to GPU, and the system memory to GPU, far outweighs the data coming from the GPU to CPU. The performance of the GPU, and therefore the quality of the gaming experience, is directly proportional to the number of frames the GPU can process per second. The data transfer bandwidth between the CPU/System Memory and the GPU plays a crucial role in this performance. If the interface between the CPU and GPU is slow, this data transfer can be a bottleneck that will hurt performance. The pace of innovation in this interface (ISA, PCI, AGP, PCIE 1.0, PCIE 2.0, PCIE 3.0) has been brisk. A typical gaming system today has bandwidth of up to 4 Gbytes/Second.
The nature of the CPU-GPU and the GPU-Display interface has required that the CPU, GPU and Display be part of the same system to guarantee the best performance. This limitation has implications for system design, such as power consumption, size, portability, cooling requirements and noise.
For these and other reasons, there is interest in the graphics community to find ways physically to separate the CPU, GPU and Display, in a way that does not require re-writing of applications. Possible solutions range from physical separation at the electrical level, to software solutions that operate at higher levels.
The problem becomes even more complicated in a networked environment, in which the user may be remote from a server that houses the main program which generates the graphics. For example, many of the most popular modern applications are interactive multi-user games applications, in which the users' computing systems are connected by a network to a centralized game program on a remote server. The user interacts with the game program by providing inputs from the user's local computer system, but the visual contents that are seen by the user are instituted by the operations of the remote game program on the server, which are then locally displayed to the user on the user's local display device.
One possible approach to implement this type of networked system is to require the CPU and GPU at the server to generate and render video data at the server-side that will be sent to the client computer, and which will utilize the video decoder at the client to be displayed at the client-side. This approach is shown in FIG. 1, in which the CPU and GPU processors 102a located at the server 108 take on the entirety of the work needed to render the display graphics, so that only a stream of video display pixels 110 are sent to the client to be displayed by the video decoder 106a at the client.
Alternatively, remote rendering may be employed to off-load some of the rendering workload to the client. This approach is shown in FIG. 2, in which the CPU 102b at the server 108 is responsible for operating the game logic, but it is the GPU 106b at the client that takes care of processing graphics data 112 sent from the server 108 to locally render visual graphics to be displayed to the user.
Each of these approaches has its own specific advantages and disadvantages. For example, if the local GPU is excessively underpowered (e.g., inadequate processor speed) or has insufficient system resources (e.g., insufficient memory), then the display performance using the approach of FIG. 2 may be much less desirable than the approach of FIG. 1. On the other hand the approach of FIG. 2 may provide much better performance than the approach of FIG. 1 under certain circumstances that can take advantage of the local GPU, e.g., display frames which do not undergo many changes will require much less network bandwidth in the approach of FIG. 2 compared to the approach of FIG. 1. While each approach has its own set of advantages compared to the other, there are no existing systems that can combine the best advantages of both into a single system.